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Abstract 



Dynamical zeta functions are expected to relate the Schrodinger operator's spec- 
' trum to the periodic orbits of the corresponding fully chaotic Hamiltonian sys- 

. tern. The relationsship is exact in the case of surfaces of constant negative 

' curvature. The recently found factorisation of the Selberg zeta function for the 

. modular surface is known to correspond to a decomposition of the Schrodinger 

operator's eigenfunctions into two sets obeying different boundary condition on 
Artin's billiard. Here we express zeta functions for Artin's billiard in terms 
of generalized transfer operators, providing thereby a new dynamical proof of 
the above interpretation of the factorization formula. This dynamical proof is 
then extended to the Artin-Venkov-Zograf formula for finite coverings of the 
O , modular surface. 

! 1 Introduction 



Dynamical zeta fuctions for flows 4't '■ M — > M were introduced in the sixties by 
Smalc [0 after he learned from Sinai about such an interpretation for Selberg's 
I zeta fuction |Q in terms of the geodesic flow on a surface of constant negative 

curvature defined by a discrete group acting on Poincarc's hyperbolic upper half 
plane the so called Fuchsian group. It turned out that Ruelle's and Bowen's 
thermodynamic formalism provides a new approach for understanding the 
properties of these functions completely different from Selberg's original one, 
which was based mainly on his famous trace formula In the new approach 
the dynamical zeta function gets expressed in tems of Fredholm determinants of 
so called transfer operators [|| . These operators haver their origin in the transfer 
matrix method of statistical mechanics and were used in the early days of the 
thermodynamic formalism to characterize the ergodic properties of dynamical 
systems j?]. 
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Since in the case of the geodesic flow on surfaces of constant negative cur- 
vature there exists a close connection between the nontrivial zeros of the cor- 
responding dynamical zeta function and the spectrum of the free Schrodinger 
operator on the surface, the transfer operator, which is a purely classical object, 
when analytically continued to complex temperatures, hence determines also 
the quantum properties of these systems ||] , |^ . One could then expect that 
something like this could be true also for general chaotic Hamiltonian systems at 
least in the semiclassical limit of the systems' quantization. Indeed a heuristic 
approach to this question was recently provided in where the quantization 
condition is again expressed in terms of a Fredholm determinant of a so called 
quantum operator, which coincides with the aforementioned transfer operator at 
least in the case of the geodesic flow on a compact surface of constant negative 
curvature ||l0|| . 

Not so much is known for such surfaces in case they are not compact, but 
have a finite hyperbolic area | [Tl| ]. Among them the modular surface, defined 
through the action of the modular group SL{2, Z) on the Poincare half plane 
K = {z = X + iy : y > 0} plays a special role: for its geodesic flow the above 
theory works perfectly and the dynamical zeta fuction can be expressed again in 
terms of the Fredholm determinant of a transfer operator [pl]| , providing at the 
same time a much easier and shorter approach to Selberg's results for such non- 
compact surfaces of constant negative curvature. Furthermore, an interesting 
factorization formula of the dynamical zeta function follows from the transfer 
operator approach, closely related to the description of the geodesic flow on the 
modular surface in terms of the symbolic dynamics of Series resp. Adler 
and Flatto |l^. The conjecture in namely, that this factorization corre- 
sponds to a decomposition of the spectrum of the free Schrodinger-operator on 
the modular surface into even respectively odd eigenfunctions under reflection 
of its fundamental domain on the imaginary axis, was proved recently injlj] by 
using number theoretic methods. A formal argument was given also in |lCl| . 

In the present paper we give a rigorous proof of this conjecture based com- 
pletely on dynamical properties of the flow and its relation to the Artin billiard. 
It turns out that our proof shows at the same time that the factorization of the 
Selberg function for the geodesic flow on the modular surface can be interpreted 
as an extension of the Artin- Venkov-Zograf factorization ||l5| for the Selberg 
function for normal subgroups F' of Fuchsian groups F in terms of Selberg func- 
tions with unitary representation of this group F. Indeed the modular group 
SL{2, Z) is a normal subgroup of GL{2, Z) and the Selberg functions of the Artin 
billiard with Neumann resp. Dirichlet boundary conditions are just generalized 
zeta functions of the latter group corresponding to the two one dimensional uni- 
tary representations of GL{2, Z) with kernel SL{2, Z). We then show how our 
approach to the factorization formula for the modular surface can be extended 
immediately to any finite covering of the modular surface, defined by a normal 
subgroup F' of the modular group SL{2, Z), and results in a simple dynamical 
proof of the Artin- Venkov-Zograf formula. An interesting case is the principal 
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congruence subgroup r(2) = {g £ 5L(2,Z) : g mod 2 = 1} which defines a 
sixfold covering of the modular surface. Its geodesic flow is closely related to 
certain approximate solutions of the so called mixmaster universe [|l6| . 

In a first section we determine the symbolic dynamics of the billiard flow on 
Artin's billiard from the one determined by Series resp. Adler and Flutto 
some time ago for the geodesic flow on the modular surface. This allows 
us to express the dynamical zeta functions for the billiard flow in terms of 
Fredholm determinants of the generalized transfer operator for the Gaufi map. 
We show that the product of these two functions gives exactly the dynamical 
zeta fuction for the geodesic flow on the modular surface. Using well-known 
results of Venkov this then proves our conjecture relating the factorization to 
the even and odd spectrum of the Schrodinger operator. Next we show that our 
factorization can be regarded also as an extension of the Artin-Venkov-Zograf 
formula for subgroups of Fuchsian groups to the case of the subgroup SL{2, Z) 
of the group GL(2,Z). 

In a last section we argue how our approach can be used to provide a dy- 
namical proof of the Artin-Venkov-Zograf formula for flnite coverings of the 
modular surface. These authors used in their work the special group theoretic 
structure of the corresponding Fuchsian groups whereas ours is mainly of dy- 
namical nature. 

Work of the authors on this paper started during a common stay at the 
Institute for Solid State Physics of Prof. P. Szepfalusy in Budapest financed 
through the German-Hungarian exchange program. The results of this paper 
were announced by one of us at the workshop "Symbolic Dynamics" at the 
MSRI at Berkeley (Calif.) in Nov. 1992. 



2 Dynamical zeta function for Artin's billiard 
2.1 Geodesic flow on the modular surface 

The Poincare half plane Ji — {z — x + iy : y > 0} is shown in figure ^. Due to 
it's metric {dx'^ +dy^)/y'^ the geodesies are circles (or straight lines) orthogonal 
to the X-axis. The modular surface is defined by identifying points z, Tz = z + 1 
and Qz = —1/z. T and Q are the generators of the group 

P5L(2,Z)-5L(2,Z)/{+l,-l} (1) 

acting on H as 

nz + B 

{/:H^H|/(z) = — ^;a<5-/37 = l;a,/?,7,^eZ} . (2) 

The fundamental domain {z G H : |z| > 1; |5Rz| < 1/2} tiles the plane H under 
the action of this group. Artin's billiard is the part of the fundamental domain 
with a; > 0. At intersections with the boundary of this domain the geodesic 



3 



flow of the billiard is reflected while the geodesic flow of the modular surface is 
mapped under Q or T or T^^ to an equivalent point of the fundamental domain 
of the modular group SL{2, Z). 

The symbolic dynamics of the geodesic flow on the modular surface was 
investigated by Adler and Flatto resp. Series They introduced a 

Poincare section, which can be finally induced on I2 x Z2. The coordinates 
G ^2 X Z2 of this induced Poincare section are determined from the 
orbit's 7 forward and backward intersections x^^o ^'iid a;_oo with the x-axis. 
Immediately before the intersection with the unit circle we have a;_|_oo G ("li 1) 
and: 



X 

4, 



= l-T+ool e(0,i) (3) 
= l/|.T_oo| e (0,1) (4) 
= sgn(a;+oo - a;-oo) e {-1, 1} ■ (5) 



In these coordinates the Poincare Map T takes the simple form • 

nx,i^^^)-iToix),j^y~e) (6) 

where [x] denotes the integer part of x and Tq is the Gauf5 map on the unit 
interval: 

TGix)--^l/x mod 1 ,X7^0 . (7) 

The x-component of the direction of motion changes its sign e at every intersec- 
tion with the unit circle. Thus the number of these intersections along a periodic 
orbit is equal to the number of Poincare mappings. Enumerating the monotonic 
branches of the Gaufi map with the natural numbers one gets a complete and 
exact symbolic dynamics of the Gaufi map. This generalizes to a symbolic dy- 
namics of the Poincare map T. Thus all the periodic orbits 7 of the modular 
surface are classified. Their length ^(7) was determined by PoUicott |^ as a 
sum over all their intersections with the Poincare section: 

^(7) = ^r(x„^„e.) (8) 

i 

r{x,'4',e) r{x) ■■=\n\TQ{x)\ ■ (9) 



2.2 Periodic orbits of the modular surface and the Artin 
bilhard 

Let us denote the periodic orbits of the geodesic flow on the Artin billiard by -fb 
and on the modular surface by 7^- We will derive a relationship between their 
lengths and numbers. 

This relationship depends on the symmetry properties of the orbits under 
the reflection at the y-axis J{x, y) = {~x, y). According to definitions (^) to (||) 



4 



this reflection acts on the Poincarc section as: 



(10) 



Applying this reflection J we can transform periodic orbits "fm of the modular 
surface into periodic orbits jb of the Artin billiard and vice versa. Any prime 
periodic orbit 7^ of the modular surface is cut into segments by its intersec- 
tions with the y-axis and the boundary of the fundamental domain. On these 
segments it is a simple geodesic flow. Applying J to every second segment we 
get an orbit, that fulfills the boundary conditions of the orbits for the Artin 
billiard. Starting from a point P with positive .x-coordinate in the direction n 
of 7m and transversing the orbit 7^,, we thus construct an orbit 7b of the Artin 
billiard, which is periodic. The prime periodic part of 7b is a unique function 
G{"/m) of the prime periodic orbit "fm- Only 7„i and its mirror image J ^m, 
but no other prime periodic orbit of the modular surface, yield the same orbit 



Thus we have constructed a map G from the set Mm of prime periodic 
orbits on the modular surface to the set Mb of prime periodic orbits in the 
Artin billiard. is divided into symmetric = {7^ e M^l J 7^ = 7^} 
and asymmetric Af," = {7,„ e M^l J 7^ 7^ 7m} orbits. The map G is injective 
on and two- to-one on M^. 

In a similar geometric way we can construct the inverse of the map G. 
Any prime periodic orbit 7b of the Artin billiard is cut into segments by its 
reflections from the wall. Applying J to every second segment we get an orbit 
7to, which fulfills the boudary conditions of the modular surface. Starting at P 
in the direction n of 7b and transversing 7b once, we thus construct an orbit, 
which ends either at {P,n) or at (J P.,Jri), depending on the number n(7b) of 
intersections with the unit circle: 

1. n(7b) even 7^ ends at (P,n). Then 7^ := 7m is a prime periodic orbit 
with the same length I and the same number n of intersections with the 
unit circle as 7b. It is different from its mirror image J 7m. 

2. n(7b) odd =4> 7^ ends at {J P,Jn). Then we create an orbit 7™ by 
concatenating J 7™ and 7™. This doubles the length: ^(7m) = 2 • ?(7b) 
and the number n of intersections with the unit circle: n(7m) = 2 • n(7b). 
The orbit 7m ends at (J^ P_, n) = {P.,n), thus it is prime periodic. It 
is also symmetric: jm = J Im- 

As there is no third possibihty, all orbits 7b G G{Mm) belong to the first case, 
all orbits 7b £ G(M^) to the second. It follows that G(M^) and G(M^) are 
disjoint and that G is surjective: Mb = G(M^) U G(M^). 



G(7m) = G(J7m). 



1. n(7b) even <^ 7b G G(M» ) 



^({76}) = {7m, J 7m} 



(11) 
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Klb) - /(7m) (12) 

n(7&) = n(7„) (13) 

2. n(76) odd ^ 7b e G(M^) 

^G-i({74) = {7m} (14) 
/(7b) = /(7m)/2 (15) 
n(7b) = n(7™)/2 (16) 

2.3 Periodic orbits of the Poincare map and the Gaui3 map 

Let us denote the periodic orbits of the Poincare map T by 7p and of the GauB 
map Tg by jq. If the length of an periodic orbit 7 is n, then the geometric 
length of the corresponding orbit of the geodesic flow is: 

n 

/(7) = E^(^') ' (17) 

i=l 

where r{x) was defined in equation (^). In analogy to the last section we divide 
the set Mp of prime periodic orbits of the Poincare map T into symmetric 
Mp = {jp £ Mp\J 7P = 7p} and asymmetric = {7P G Mp\J ■yp ^ 7p} 
orbits. 

As the first component Tg{x) of the image T(x, V'l e) does not depend on ip 
and e, there is a trivial projection (x, ip, e) > Xj which maps any prime periodic 
orbit 7p of the Poincare map T onto a periodic orbit 7^ of the Gaufi map Tq. 
The prime periodic part of is a function H{^p) of the prime periodic orbit 
7P. 

On the other hand, if 7^ is a prime periodic orbit {xijiGz; Xi+n — Xi! 
Xi+i = Tcixi) of length n(7G) of the GauB map, we can find corresponding 
orbits of the Poincare map. The coordinate Tpi is fixed uniquely by the periodic 
continued fraction 

V-.-TTT 1 / 1 ■ (18) 

Again we have to distinguish two cases: 

1. ni^c) even (xo,^Oil) and (xoiV'Oi— 1) are starting points of two dif- 
ferent prime periodic orbits 

{lP.Jlp}^H-\{-iG}) (19) 

of length 

n{lp) = n{J 7p) = n(7G) • (20) 
The corresponding geometric lengths are also equal: 

1{1P)=1{J -1p) = 1{ig) ■ (21) 
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2. n{'~^G) odd =^ (xo,V'0)l) and (xcV'o,— 1) belong to the same symmetric 
prime periodic orbit 

M = H-\{^g]) (22) 

of length 

n(7p) = 2 • n(7G) • (23) 
The corresponding geometric length, too, has doubled: 

Z(7P) = 2 • Z(7g) . (24) 

Thus the set Mg of prime periodic orbits of the Gaui3 map Tg is split into two 
disjoint parts if(M^) and H{MI,). 

2.4 Pcirtition sums 

Now we have related the prime periodic orbits of the Artin billiard to those 
of the modular surface, those to the ones of the Poincare map and finally the 
latter ones to those of the Gaufi map. In order to determine dynamical zeta 
functions we have to calculate sums over all prime periodic orbits of a quantity 
(j), that depends on the geometric length ^(7) of the orbits. The sum for the 
Artin billiard is: 

E '^('(^'')) = E '/'('(7b)) + E ^('(7f)) 



I- E mim))+ E ^il-Kim)) 

I- E m7p))+ E -^(^'(7P)) 



2 

= l'^- E m7G))+ E <^(^-2-K7g)) 

7G6-fi"(M») 7Geff(M|,) 

= E <^('(7g)) (25) 
7GeMG) 

equal to the sum for the Gaufi map. 

In the same way one shows, that the number ^(75) of intersections of an 
Artin billiard's orbit with the unit circle is equal to the length 71(70) of the 
Gaufi map's orbit: 

1. n(7G) even => 71(70) = "(tp) = «(7m) = "■(ifc) 

2. n(7G) odd ^ 71(70) = n(7p)/2 = n(7m)/2 = n(7b) 

Thus we can calculate the dynamical zeta functions for the Artin billiard as 
easily as those for the Gaufi map. 
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2.5 Dynamical zeta functions 

When calculating the Ruelle zeta function of a billiard with Dirichlet bound- 
ary condition we have to include the phase factor exp(7ri) — (—1) at every reflec- 



tion from a wall |22 and thus the phase factor exp(27ri) = 1 at every reflection 
from a corner in between two walls. In the Artin billiard the x-component of 
the orbit's direction changes its sign at every reflection from the lines a; = or 
X = 1/2, but it keeps it sign when reflected from the unit circle. As the total 
number of sign changes along a periodic orbit must be even, it is sufhcent to 
include the phase factor (—1) at every reflection from the unit circle, that is 
'^(7b) = TT-ilc) times. 

The Ruelle zeta function of the Artin billiard with Dirichlet boundary 
condition is defined for large enough real part as a product over all prime 
periodic orbits 7: 



Z§ ^ ^ TO 

^ -YbeMb '"=1 

= exp(- 

^ — ' ^ — ' TO 

7gGMg m=l 



= 11(1 - exp(-/3 . /(7)) ■ (26) 
This Euler product can be rewritten as a Dirichlet sum- like formula |^ : 

-j^{(3) = exp(~ ^ - • (e-W7.) . (_i)"(7.))™) (27) 

beMb '"=1 
00 

~ • (e^'^'^'^'^^ • (-1)"('>'g))'") (28) 

gGMg m=l ^ 
00 

= exp(-^-. e-'^'"^-) • (-1)") , (29) 

— n — ' 

n=l sSFixT^ 

where FixT^ is the set of fixpoints of Tq and 

Z„(x) :=^r(T^(a:)) . (30) 

i=0 

The sum of exponentials can be calculated by the transfer operator method |^ . 
The transfer operator L/3 is the generalised Frobenius Perron operator of the 
GauB map: 

i./<») E (31) 

Tciw) = X 

Y f{w)-e-P^'^^^ . (32) 



Tci: 



W) — X 
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The result is: 



n-l 



^ ^-PU^) = ^ l[cM-PriT^i^)) (33) 

xeFixTg xeFiKTg i=0 

= Tr(i^) - Tr((-L^+i)") . (34) 

Inserting this into equation ( p9|) we get: 

1 °° 1 

-^(/3) = exp(-^-.(-l)".(Tr(L^)-Tr((-L^+i)"))) (35) 

det(l + L/3) 



det(l - L^+i) 
The Selberg zeta function hence is given by: 



(36) 



oo 



fc=i 



In the same way, only replacing the phase factor (—1) by 1, one gets the Ruelle 
zeta function of the Artin billiard with Neumann boundary conditions: 

1 det(l-L^) 
Z^^^^-det(l + L,+i) • 

The Selberg zeta function for Neumann boundary conditions then is: 

Z?,«.d.t(l-Lrf^nS[^ . (40) 

In jl^ Bogomolny and Carioli conjectured, that the Selberg zeta functions for 
the two cases of Dirichlet and Neumann boundary conditions should be equal 
to det(l + Lp) and det(l — LgV This conjecture is violated by the additional 
factors in equations (|3|) and (po|). 

The poles and zeros on the line K/3 =1/2, which determine the spectrum of 
the Laplace-Beltrami operator are, however, not affected by these additional 
factors: It follows from a result by PoUicott in that for 5R/3 > 1 all the 
eigenvalues of have a modulus smaller than 1. Thus dct(l + L/j+fc) and 
det(l — L/3+fe) have no zeros for 5ft/3 — 1/2 and fc > 1. In it was shown, 
that /3 = (1 — j)/2; j e Nq are the only poles of Lp. Thus det(l + Lp^u) and 
det(l — i/3+fc) have no poles for K/3 =1/2 and fc > 1. 
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Finally, multiplying the Selberg zeta functions for Dirichlet and Neumann 
boundary conditions, we get the Selberg zeta function for the modular 
surface Q as: 

Zfip) = Z^{fl).Z^{P) (41) 
= det(l-L/3)-det(l + L^) . (42) 



3 The Artin— Venkov— Zograf factorization for- 
mula 

In ijl^ Venkov and Zograf proved a remarkable factorization formula for the 
Selberg zeta function for normal subgroups V of a Fuchsian group F with finite 
factor group F/F'. This formula is well known in algebraic number theory 
and was found for number theoretic zeta functions by Artin and Tagaki |p^ . 
Starting from the relation of the spectra of the free Schrodinger-operator on the 
corresponding surfaces for the groups F and F' they showed that the Selberg 
zeta function for the subgroup F' can be simply expressed as a product of 
zeta functions for F with all finite dimensional unitary representations of this 
group which have F' in their kernel. To be more precise, denote by x*(r/r') 
all inequivalent unitary irreducible representations of the factor group F/F' = 
• F' : 5 G F}, whose elements we denote by {g}. If x G x*(F/F') then x 
obviously defines also a unitary representation x of the group F by 

xig) = xig ■ r') (43) 

For 5 e F' this gives 

xig) = xig ■ r') = x(i • r') = i (44) 

and hence F' e kernel x- 

Consider next the generalized dynamical zeta function Zg{f3,x) of the 
geodesic flow on the surface determined by the group F |Q : 

oo 

X) n n (l " x(P,)e-(''+'^-)'(-)) , (45) 

where denotes an element of F which fixes the closed orbit 7, that means 
P^Xoo — Xoo, P-yX-oo — X-oo, if 7 = (a^-oo , 2^+00 ) IS this orbit in the Poincare 
half-plane H and x is a representation of F as discussed before. 

Obviously for the trivial one dimensional representation xo with Xo{g'^') = 1 
we also have Xo(.9) = 1 for all .g G F and hence 

Z^(/3,xo) = ZliP) = n ft (l - e-('^+'')'(-)) (46) 

7 fc=0 
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is the ordinary dynamical zeta function for the geodesic flow on the surface H/F 



The proof by the above authors uses the specific nature of the algebraic struc- 
ture of Fuchsian groups. However, it turns out that this formula has an almost 
trivial interpretation when the dynamics of the involved geodesic flows is taken 
into account. This approach works at least for compact sufaces of constant neg- 
ative curvature and their finite sheeted coverings. More interestingly, it works 
also for the modular surfaces, that means the surface defined by the modular 
group SL{2, Z) and its finite coverings. It is expected that the same arguments 
can be applied to general constant negative curvature surfaces with finite vol- 
ume, as soon as the thermodynamic formalism approach to their dynamical zeta 
functions has been worked out |pl| . Interestingly enough, this approach even 
extends to a case strictly speaking not covered by the Venkov-Zograf paper, 
namely the group GL{2, Z) of all 2 * 2 integer matrices with determinant ±1 — 
which is not a Fuchsian group. 

We will show now, that the factorization of the dynamical zeta function for 
the modular surface as dicussed in the first sections of this paper can indeed be 
interpreted in this way. To see this, remember the transfer operator of the 
geodesic flow on the modular surface [O: 



The group SL{2,Z) is a normal subgroup of GL(2,Z) and Gi(2, Z)/S'L(2, Z) 
has just two elements which we denote by {g} — ±1 corresponding to the 
two classes g ■ SL{2,Z) with g G Gi(2,Z) and det^ — ±1. The group 
GL{2,Z)/ SL{2,Z) has just two flnite dimensional irreducible unitary repre- 
sentations X17X2J both one dimensional with 




(47) 



xex*(r/r') 




(48) 



Xi(ff-5i(2,Z)) = l 



(49) 



and 



X2{g-SLi2,Z))^detg (50) 



for all g E GL{2, Z) |]T^. The corresponding representations Xi, X2 of GL{2, Z) 
with SL{2, Z) in their kernel are 



XI (5) = 1 (51) 



and 



X2(5) = det5 



(52) 
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The above transfer operator Lp can be rewritten also as follows: 

oo . . 2/3 , 

hf{z^{9}) = Hi-^) f{j^AJQT-"]{9}) (53) 

n— 1 ^ 

= T.[-^) X.({T"Q^})/(^,{.}) (54) 

n— 1 ^ ^ 

where 

XL ■■ GL{2, Z)/SL{2, Z) ^ Aut(C(GL(2, Z)/5L(2, Z))) (55) 

denotes the so called left-regular representation of the group GL{2, Z)/SL{2, Z) 
on the two dimensional space of complex valued functions on the group 
GL{2, Z)/SL{2, Z) defined quite generally for any finite group G as fl9| : 

XL{9')f{9) = f (g'^'g) for g,g' eG (56) 

and / G C(G), the space of complex functions on G with dim 0(6") = 
order of G. 

In equation ( js^ ) J denotes the reflection Jz = —z* and T and Q are the 
generators Tz = z + 1 resp. = — 1/z of the group PSL{2,Z). They are 
given by the corresponding matrices in GL{2, Z). We also made use of the fact 
that {JQT-"}{5} = -{g} for aU n and aU {g} = ±1 since det(JQT-") = -1 
for all n. 

To understand expression ( |5^ ) better remember that the Poincarc map for 
the Artin-billiard was simply 

P(.,.)=(Ta.,^^) . (57) 

Consider then a geodesic on the Artin billiard starting in the point x, y of 
the Poincare section. On the modular surface this point is covered by the 
two points {x,y,{g}) with {g} = ±1 G GL(2,Z)/SL(2,Z). When starting 
in one of them, say {x,y, 1), the point will come back to a lift of the points 
P{x, y) in the Poincare section of the Artin billiard. From the symbolic dy- 
namics of Series et al. [Q, it follows that the Poincare map P just 
corresponds to the map JGT^^ of the endpoints a;_oo and x+oo of the cor- 
responding half circle 7 in the upper half plane and hence the geodesic ar- 
rives in the point {P{x,y),{JQT^^^}) — {P{x,y), —1), which then defines the 
Poincare map for the geodesic flow on the modular surface. The left regular rep- 
resentation xl can be decomposed completely into its irreducible components 
[Eol which are just all the finite-dimensional irreducible unitary representations 



X*{GL{2,Z)/SL{2,Z)) and therfore xi and X2- It is known |19 that each of 



these representations occurs in xl just as many times as given by its dimension, 



12 



hence exactly once. The transfer operator L/3 hence can be written as 

Lib = i/3,xi ® ■^/3,X2 (58) 

00 . s 2/3 

With i;,,^ J(z) = E ^ /(^) (59) 

n— 1 ^ ' 

00 . s 2/3 , 

and L;3,^,/(z) - /( ) = _i /(^) (60) 

n— 1 ^ 

since X2{T"QJ) = -1 for all n . (61) 

It is then straightforward to show, that the zeta functions for the Artin 
billiard with Neumann and Dirichlet boundary conditions coincide with the 
zeta functions for Gi(2,Z) with representations xi a-nd X2- 

Z^{/3) = Z^^(2'^)(/3,xi) (62) 

andZ^if]) = Z^'^^^'^\p,X2) (63) 

and our factorization 

Zl''^'^^\p) ^ ZiiP) ■ Z^ iP) (64) 

coincides indeed with the Venkov-Zograf formula for SL{2, Z) when considered 
a subgroup of GL{2, Z). 

The above sequence of arguments extends immediately to any finite sheeted 
covering of the modular surface. If T' denotes the corresponding subgroup of 
the group F = SL{2, Z), then the covering group of H/F' with respect to H/F 
is just F/F'. The Poincare map for the geodesic flow on the covering surface 
can be constructed from the Poincare map T : /2 x Z2 — > /2 x Z2 (H) of the 
modular surface in complete analogy to our procedure in going from the Artin 
billiard to the modular surface: in the present case every point in the Poincare 
section of the geodesic flow on the modular surface is covered by d = [F : F'] = 
#{{5} e F/F'} points given by {x, y, e, {g}), {g} G F/F'. A geodesic starting at 
the point [x, y, e) in the Poincare section of the modular surface and arriving 
at T{x, y, e) gets therefore lifted to a geodesic on the covering surface starting 
in {x,y,e,{g}) and arriving at the point [T{x,y,e),{QT~^'^}{g}) in the lifted 
Poincare section, where n — [l/x\. This gives the following Poincare map: 

rr'(a;,y,e,{g}) = (f(x,y,e),{gT--}{5}), n=[l/x] . (65) 
The generalized transfer operator hence has the form: 

L^7(^, e, {g}) = (^) f -e, {r"g}{.g}) . (66) 

n—l \ ^ ^ ^ 
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Introducing again the left regular representation xl of the group T/F', on the 
space of complex functions on T/V we can write also as 



L^7(z,e,{g}) = £ 



2/3 



Xl({QT— })/ 



1 



z + n 



(67) 



The left regular representation xl decomposes again as 



XL= n,Xi 
x.6x*(r/r') 



(68) 



with Hi being the dimension of the irreducible representation Xi of T/V . This 
shows that the transfer operator for the geodesic flow on the covering surface 
H/r' of the modular surface can be rewritten in the appropriate basis of the 
space of complex functions on T/V as 



2/3 



(69) 



/ xi(Qr-"^) 



\ 



But this shows that 



^ni -times 



>nj: -times 



(70) 



where 



1 



2: + n 



2/3 



x^iQT-nf 



1 



z + n 



(71) 



is the transfer operator for the geodesic flow on the modular surface with unitary 
representation Xi of the group T = SL{2, Z) with V in its kernel. From this the 
Venkov-Zograf formula (p7|) follows immediately, since 



Zl.{P,x^)^det{l-Ll^J 



(72) 



as can be verified by standard arguments. 
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Of special interest is the case of the principal congruence subgroup r(2) 
of the modular group with 

r(2) := {g e SL{2, Z) : g mod 2 = 1} . (73) 

Since 

5L(2,Z)/r(2)^5L(2,Z2) = ^3 , (74) 

where S3 is the group of permutations of three elements, H/r(2) is a sixfold 
covering of the modular surface. Indeed, its fundamental domain can be chosen 
asi: 

f' = {z e H : < |5R2| < l,|z± 1/2| > 1/2} . (75) 

T(2) 

Its zeta function Zg (/?) can be written as 



(/?) = [P, xi) ■ (/3, X2) • (zf (/3, X3)) ' (76) 



where xi denotes the trivial representation of 53, X2 is the one dimensional 
representation with X2{t) — il if is an even resp. odd permutation. The 
two dimensional representation ^3 is given by the rotation of the plane by the 
angle ±27r/3 respectively the reflection on the y-axis: x —>■ —x and y —>^ y 
[ [l9| . Since ±Q and ±T generate SL{2, Z) any of its representations is uniquely 
determined by giving the representation of Q and T. For the case r(2) we need 
the representations which have r(2) in their kernels and these are the following: 

XI (5) = lforall5e5i(2,Z) (77) 

X2iQ) = -I (78) 

X2(T) = -1 (79) 

, - f -cos27r/3 sin27r/3 \ , . 

^ [ sin 2^/3 cos2vr/3 ) ^^^^ 

X3(T) = ( "J ? ) , (81) 
where we have, according to (ll, identified Q=('? MandT ' ^ ^ 



^ I J V 1 

with matrices in SL{2, Z). 

The Venkov-Zograf formula then allows to determine the spectrum of the 
free Schrodinger operator on the surface H/r(2) through the spectra of the 
Schrodinger operators on the modular surface with the boundary conditions 

/(5^)=X.(.9)/(^) (82) 

with g the two generators T and Q, what one expects also from purely spectral 
reasons. These spectra are again determined by the nontrivial zeros of Zg {f3, Xi) 
on the line K/3 =1/2 and hence by those /3- values on this line for which Li^^-^. 
has A = 1 in its spectrum. 
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Figure 1: The shaded area is the domain of the Artin bilHard. It is half as big 
as the fundamental domain of the modular surface. The geodesic flow takes 
place along vertical lines or halfcircles centered on the x-axis. Examples are the 
halfcircles 71, 72 and its reflection image J72. 

Comment for users of the automated preprint bulletin board: This figure has 
not been added to the file, as it is purely introductory. Pictures of the symbolic 
plane can be found for example in the book by Gutzwiller or Terras. 
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